Label2Label: A Language Modeling Framework for Multi-attribute Learning

AbstractObjects are usually associated with multiple attributes, and these attributes often exhibit high correlations. Modeling complex relationships between poses a great challenge for multi-attribute learning. This paper proposes simple yet generic framework named Label2Label to exploit the attribute is first attempt prediction from perspective of language modeling. Specifically, it treats each label as “word” describing sample. As sample annotated labels, “words” will naturally form an unordered but meaningful “sentence”, which depicts semantic information corresponding Inspired by remarkable success pre-training models in NLP, introduces image-conditioned masked model, randomly masks some tokens “sentence” aims recover them based on context conveyed image features. Our intuition that instance-wise relations well grasped if neural net can infer missing remaining hints. conceptually empirically powerful. Without incorporating task-specific prior knowledge highly specialized network designs, our approach achieves state-of-the-art results three different learning tasks, compared customized domain-specific methods. Code available at https://github.com/Li-Wanhua/Label2Label.KeywordsMulti-attributeLanguage modelingAttribute